Mimicking Human Vision in Hmds

Head mounted displays (HMDs) have disappointed real world users in their inability to live up to over-hyped expectations. This does not, however, mean that HMDs are useless. While still technologically lacking in some areas, appropriately designed HMDs can be extremely useful tools. We will look at the limitations of current HMDs and ways around them. Rather than approach the problem from the optical, electrical and mechanical engineer’s point of view, we will approach it from the physiology point of view, answering the question; what is needed to create a useful HMD? The paper is divided into two separate sections. The first, is a description of the performance of the human visual system. The second, addresses how designers attempt to mimic the human visual system in an HMD. This second section will discuss applications that need the specific performances described in section one, current solutions to those needs and finally ideal solutions not yet implemented. Finally, a summary of these findings is presented in a table format. PERFORMANCE OF VISUAL SYSTEM I. Field of View Each eye can see 140 horizontally and 110 vertically. Most viewing is looking strait ahead and when combining the two eyes as a team, as most of us do, the horizontal FOV is about 195 . A landscape image is the result of these combined images and displays, TV, HMDs exploit this comfortable format. However, when an HMD is used in a walking or mobile situation, the rules change. For balance, the wearer needs a horizon. Outside, vanishing points are a necessary. While indoors, a floor edge must be visible. The indoor HMDs requires a large FOV since a distance of 2 meters from a wall requires a downward visibly of about 45°. A smaller FOV is acceptable in many stationary uses; however, in mobile or walkthrough situations, very large FOV is critical. A minimally acceptable 60° vertical FOV and minimum 75° horizontal is required for mobile uses. II. Resolution The eyes tend to work hard to focus a blurry or low-resolution image. When the image displayed is of low quality, out of focus or too few pixels, the eye strains and becomes tired. FOV does not contribute to this. In the center of the eye, the maximum discernible resolution is 1 arcsec. However, 1 arcmin is completely acceptable for long term use and 3 arcmin is acceptable for shorter wear times. During times of moderate movement, 2-4°/ second, resolution can be decreased about 1⁄2 in the direction of rotation without adversely affecting the HMDs wearers viewing quality. What this saves the designer is not as much in the HMD itself but in the video image creation of having to calculate or render fewer pixels. III. Color Resolution There are no HMD displays capable of displaying accurately the entire sensitivity range of the eye. However, this should not be considered a technological hindrance. The interest in most HMDs is displaying daylight (photopic) imagery, preferably in color. Some military and surveillance users require low, monochrome light levels to maintain night vision (scotopic) capability. The eye, in the photopic sensitivity region, can see an intensity variation of about 1000 to one for each of the three colors; Red, Green, and Blue. This translates to 10 bits per color or about 30 bits color depth for each pixel resolvable by the eye. Not all applications require such depth and this should be noted during design and purchase of an HMD. A unique aspect of this color depth is that it is not needed throughout the viewing area. In fact, as viewing angles extend beyond 60°, many colors can no longer be detected. As an experimental verification, while looking strait ahead, close the left eye, and pass two pens, one blue and one green, from the leftmost view point to straight ahead. Both pens appear to be gray at first. As they move closer toward the center of the FOV, the color in the blue pen becomes apparent before that in the green pen. When passing the pens back the other way, the colors will stay with them the whole way. This is because the brain has associated color to that object. Figure 1 shows the color sensitivity from the nasal out to the temporal FOV. This realization in perception may help designers of HMDs to reduce the information level in these areas without adversely affecting the wearer. IV. Accommodation/Stereo The visual cues pertaining to depth perception include occlusion, stereo, accommodation, and vergence. Of these, occlusion, closer objects blocking more distant objects, is the most important. Occlusion is the only depth cue for objects beyond 2 -3 m from the observer and is extremely important when objects, or the user, are in motion. Stereo, a different view seen by the left and right eyes, becomes the next most important cue for objects closer than 3 m. Depth through stereo can be achieved without accommodation as is the case in most HMDs and without vergence as with autostereograms. Both of these prove unsatisfactory for extended use applications because of eye fatigue. Accommodation, changing the focus of the eye for objects of varying distances, is very important for the long-term comfort of the HMD user. On average, a person changes focus several times each minute. Each re-focus exercises the eye muscles. The fatigue reported by people who stare into computer monitors all day is partially caused by keeping a fixed focus for longer than is natural. Vergence, aiming the pupils directly at an object, is not a critical cue for depth perception, but it is used for objects closer than 1 m. Discomfort occurs when the stereo cue places an object in a different location than either (or both) the accommodation or vergence cues. If an image is to look and feel real, it must contain all four depth cues. Emphasis toward each of these depth cues in an HMD design depends on application and length of time the wearer will be using the HMD. Figure 1 Angular color visibility V. Eye Characteristics The physical characteristics of the eye strongly affect the performance of the human visual system. Pupil characteristics are the most relevant to HMD designs. HMDs must account for both the size and location of the pupil of the eye. The pupil can vary in diameter from 2 mm in bright light to 7 mm in darkness. Under normal daytime indoor conditions, 4 mm is a good average pupil diameter to use in HMD design. The pupil, however, is not stationary. The eye rotates about a point roughly 12 mm behind the pupil. Easy rotation angles are ± 7.5° (H) and + 0°, 30° (V) while maximum rotation angles are ± 15° (H) and + 30°, 35° (V). Accounting for a 4 mm pupil diameter and easy eye motion means that HMD designs have to have an exit pupil located at the eye pupil which is roughly ± 5 mm (H) and + 3 mm, 9 mm (V). VI. Head Motion The head motion that an HMD will encounter depends highly on the application. Typical vertical motion is but ± 15° sitting and ±10° standing. Comfortable horizontal movement is ±45° easy and ±79° maximum and the rotational speed can be greater than 500°/ sec. However, a heavy HMD will slow this rate down. Additionally, a heavy HMD also decreases the total vertical motion. This is due to the unnatural balancing of several pounds on one’s head. People accustomed to HMD use will normally utilize a full motion pattern. Head motion for medical use will exceed the standard vertical range by looking downward at a 45° angle for extended periods. Military pilots also extend the standard head motions required including rotational speeds. Tracking is extremely important for head motion. Lag between the image displayed and the actual head orientation/location can be the greatest obstacle in creating an effective HMD based system. The motions expected in the HMD should be the basis for determining what type of tracker will be needed. When an image registered seethrough type display is needed or targeting is desired, lag must be minimized to ≤ 16ms. Although a lag of 16 ms is perceivable, most HMD / tracking / rendering systems are in the 60 – 90ms range. MIMICKING HUMAN VISION IN HMDS I. Wide FOV Wide Field of View displays are needed in walk-through systems, pilot HMDs, and games. These are all applications where data is gathered from the periphery, adding a sense of “presence”. WFOV displays require large eye movements, which necessitates large exit pupils for the optics. Most HMDs that advertise WFOV are actually around 60. This is due to that it is optically difficult to create a wrap around image from a single display device in a severely restrictive volume as an HMD. The exit pupil is too great while the F #s are too small. 1 Woodson Figure 2 Wide FOV HMD To work around these problems, one particular solution uses multiple displays with separate optics that wrap around the eye. This tiled system is designed by Kaiser Electro Optics (Figure 2). It has 6 color displays per eye providing a FOV of 150 horizontal and 50 vertical with a 40 overlap. However, its 1.5 lb. weight and complicated mechanism required to maintain the display’s position for the user make it appropriate only for laboratory use. Additionally the display has scene breaks, or mullions, between displays. While mullions are actually preferable to edge blending (even slight errors are unacceptable), this system has mullions located directly in the center of the FOV. This is annoying and unacceptable. An ideal WFOV display would have variable resolution. Putting a very high concentration of pixels directly in front of each eye and digressing as one goes towards the edges. A full periphery display, all 195 would be extremely useful in walk-through type applications. Projection systems are able to provide the WFOV immersion better than HMD historically